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Abstract 

Background: The increasing prevalence and associated cost of treating chronic obstructive pulmonary disease 
(COPD) is unsustainable, and focus is needed on self-management and prevention of hospital admissions. 
Telehealth monitoring of patients' vital signs allows clinicians to prioritise their workload and enables patients to 
take more responsibility for their health. This paper reports the results of a pilot randomised controlled trial (RCT) of 
Telehealth-supported care within a community-based COPD supported-discharge service. 

Methods: A two-arm pragmatic pilot RCT was conducted comparing the standard service with a 
Telehealth-supported service and assessed the potential for progressing into a full RCT. The co-primary outcome 
measures were the proportion of COPD patients readmitted to hospital and changes in patients' self-reported 
quality of life. The objectives were to assess the suitability of the methodology, produce a sample size calculation 
for a full RCT, and to give an indication of cost-effectiveness for both pathways. 

Results: Sixty three participants were recruited (n = 31 Standard; n = 32 Telehealth); 15 participants were excluded 
from analysis due to inadequate data completion or withdrawal from the Telehealth arm. Recruitment was slow 
with significant gaps in data collection, due predominantly to an unanticipated 60% reduction of staff capacity 
within the clinical team. The sample size calculation was guided by estimates of clinically important effects and 
COPD readmission rates derived from the literature. Descriptive analyses showed that the standard service group 
had a lower proportion of patients with hospital readmissions and a greater increase in self-reported quality of life 
compared to the Telehealth-supported group. Telehealth was cost-effective only if hospital admissions data were 
excluded. 

Conclusions: Slow recruitment rates and service reconfigurations prevented progression to a full RCT. Although 
there are advantages to conducting an RCT with data collection conducted by a frontline clinical team, in this case, 
challenges arose when resources within the team were reduced by external events. Gaps in data collection were 
resolved by recruiting a research nurse. This study reinforces previous findings regarding the difficulty of undertaking 
evaluation of complex interventions, and provides recommendations for the introduction and evaluation of complex 
interventions within clinical settings, such as prioritisation of research within the clinical remit. 

Trial registration: Current Controlled Trials ISRCTN68856013, registered Nov 2010. 
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Background 

Given the forecast increasing prevalence of chronic ob- 
structive pulmonary disease (COPD), current models of 
care provision are unsustainable and must adapt to em- 
brace prevention and self- management [1]. This poten- 
tially requires individuals diagnosed with COPD to be 
supported to manage the disease at home, thereby avoid- 
ing hospital admission and reducing healthcare costs [2] . 

COPD is characterised by progressive worsening of 
lung capacity. Patients with advanced COPD typically 
experience impaired physical, emotional, and social func- 
tioning which results in poor quality of life [3]. In the UK, 
COPD is the fifth largest cause of mortality and the sec- 
ond largest cause of emergency admissions to hospital [4]. 
COPD costs the National Health Service (NHS) over £800 
million per annum [5]. 

Remote monitoring of patients' physiology and symp- 
toms using Telehealth is considered to be highly appro- 
priate for people with COPD, a condition which is 
associated with frequent hospital admissions and high 
levels of disability and depression [6,7]. It involves the 
remote exchange of patient data (e.g., vital signs) be- 
tween a patient and clinician which can be used to iden- 
tify potential deterioration, prevent avoidable hospital 
admissions, and help improve an individual's quality of 
life [8]. Telehealth also has the potential to help patients 
improve their ability to self-manage their condition, e.g., 
through patient education [7]. 

A recent Cochrane review [7] demonstrated the poten- 
tial for Telehealth in reducing hospital admissions and 
increasing quality of life; however, in the identified stud- 
ies, Telehealth was usually delivered as part of a more 
complex package of care and, thus, it was difficult to 
separate the effect of the technology from other aspects 
of the service. The recently reported results of a large 
scale UK trial of Telehealth also demonstrated that use 
of this technology within services has the potential to re- 
duce mortality and emergency admission rates [9]; how- 
ever, the results were not definitive in that the same 
study failed to demonstrate cost-effectiveness. Neverthe- 
less, implementation of Telehealth remains a policy pri- 
ority in the UK [10] and internationally, for example, 
through the Veterans' Association in the US [11] and 
across Canada [12]. Despite the cited benefits of Tele- 
health [13], research has illustrated that patients and 
front line clinicians may not be receptive to the inter- 
vention, with the extent of staff training and support 
that is necessary to embed such technology into routine 
practice being emphasised [14]. 

This paper reports results from the pilot stage of a prag- 
matic randomised controlled trial (RCT) of a Telehealth- 
supported service and a standard service pathway 
provided through one hospital discharge service for people 
with COPD within one primary care trust (PCT) in the 



UK. Findings are reported in accordance with the Consoli- 
dated Standards of Reporting Trials (CONSORT) check- 
list for clinical trials [15]. 

Study context 

In May 2009, a PCT in the North of England introduced 
a dedicated discharge service for individuals with early 
stage COPD as defined locally". It had been identified by 
the PCT that approximately 1,200 patients with COPD 
were being discharged from local acute in-patient ser- 
vices annually. A standard (non-Telehealth-supported) 
service was established to assist patients to manage their 
illness more effectively following hospital discharge, with 
the aim of decreasing readmission rates. Two COPD 
specialist nurses, one specialist physiotherapist, and one 
community matron were employed full time within the 
service. The COPD supported discharge service received 
referrals from the local NHS Acute Care Trust via two 
routes: i) specialist COPD nurses based on respiratory 
wards who were able to refer patients directly and ii) a 
telephone referral route allowed any relevant staff mem- 
ber within the hospital to refer a patient with a diagnosis 
of COPD, even if COPD was not the primary cause of 
their admission (see below for full referral criteria). 

Criteria for referral to the supported discharge service 

• SpOa > 90% on air or pOj > 7 kPa/pH 7.35-7.45 

• Respiratory rate <25 

• Temperature <37.8°C 

• Systolic blood pressure 90-180 mm/Hg 

• Pulse 50-100 BPM 

• Orientated and alert/able to give consent 

• Safe discharge environment 

• Between 1 and 3 previous admissions (including the 
current admission) in the previous 12 months from 
the current date of discharge where COPD is the 
primary or secondary documented reason for 
hospitalisation 

The service involved six home visits over the 8-week 
time frame, resulting in a conservative estimate of 
8 hours and 25 minutes of time spent with each patient 
(including clinical administration time and cancelled ap- 
pointments). It was recognised by the PCT that a service 
which was delivered entirely face-to-face was unsustain- 
able in the long term, particularly given the numbers of 
patients presenting to acute care with COPD and in the 
context of an ageing population and static NHS budgets. 
A subsequent decision was taken by the PCT to intro- 
duce Telehealth within the discharge service. The se- 
lected Telehealth system (Doc@Home'') enables the 
patient to undertake daily vital signs monitoring. If mon- 
itored signs and symptoms fall outside anticipated 
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parameters for the individual, or if the user fails to 
undertake monitoring activity, clinician alerts are gener- 
ated so that appropriate action can be taken. 

Table 1 provides an overview of the standard service 
and Telehealth-supported service. It was estimated that 
reduction of the number of home visits and inclusion of 
remote monitoring of patients' vital signs could reduce 
the average time spent on each patient to 5 hours 30 mi- 
nutes, thus better utihsing staff resources and reducing 
costs. Telehealth equipment was to be provided to pa- 
tients for 8 weeks, in accord with the overall service 
offer. Agreements were established between the PCX, 
Local Authority, and equipment provider for the instal- 
lation, delivery, alert management, and de-installation of 
the equipment. Both the standard service and the 
Telehealth-supported service are free at the point of de- 
livery for its users. 

The research programme 

The overall research questions (agreed with the PCT ser- 
vice commissioners and managers) were as follows: 

• Does a Telehealth-supported discharge service 
decrease hospitalisations compared to the standard 
service? 

• Does a Telehealth-supported discharge service result 
in improved quality of life for people with COPD 
compared to the standard service and does this 
change over time? 

• Does a Telehealth-supported discharge service 
reduce use of NHS resources compared to the 
standard service? 

Study design was informed by the Medical Research 
Council guidance for evaluating complex interventions 
[16]. It involved a feasibility study (reported elsewhere) 
which investigates the practicalities of undertaking the 
research, e.g., flow of referrals into the discharge service 
and clinician/patient engagement levels. The feasibility 
study was followed by a pilot randomised controlled trial 
(RCT) to finalise study methods. The intention was to 



then continue to a full RCT, as described in the pub- 
lished protocol [17]. 

The specific objectives of the pilot randomised con- 
trolled trial were to i) test the trial methodology, namely 
recruitment, randomisation, intervention implementa- 
tion, and outcome measurements; ii) estimate the sam- 
ple size required for a full trial, through analysis of data 
on patient contact with other services, including hospital 
admissions; and iii) conduct a preliminary evaluation of 
the cost-effectiveness of the Telehealth intervention 
through analysis of healthcare usage, patient contact data, 
and quality of life data. 

Methods 

The study reported in this paper involved a pragmatic 
two-arm pilot RCT informed by the findings from a prior 
feasibility study. The pilot RCT followed an 'internal/ex- 
ternal' design [18] so that, if the chosen methods were 
consistently and rigorously applied, the pilot data might 
be incorporated into a full trial. 

The pilot trial was conducted over 14 months. Through- 
out the data collection period close contact was main- 
tained with all stakeholders via monthly project steering 
meetings. Records were maintained of emergent issues 
and identified solutions. The feasibility study identified 
that a target sample size of n = 60 participants, recruited 
over a 3 month period, would be acceptable for the pilot 
RCT given the number of referrals into the service and an 
estimated acceptance rate of 32% [17]. As shown in 
Table 2, potential candidates were approached by a COPD 
discharge team clinician during the first post-discharge 
home visit and were provided with study information if 
they expressed interest in participating. During the second 
visit 48 hours later, further information was provided and 
written consent then obtained from those who wished to 
participate. Random allocation to the two arms of the trial 
was generated through a web-based programme, accessed 
by the administrator for the COPD service, who generated 
the allocation online and informed the clinician immedi- 
ately following receipt of consent. Participants received 
their allocated treatment pathway for 8 weeks, with a 



Table 1 Summary of standard and Telehealth-supported services 



Timeline 


Standard COPD service 


Telehealth-supported COPD service 




1 day - First home visit after iiospital disciiarge 


Home visit 


Home visit 




3 days 


Home visit 


Home visit 




5 days 


Home visit 


Home visit 

Telelnealtin equipment installed 




2 weeks 


Home visit 


Remote review of Teleiiealtli parameters tiirougliout E 


5 weeks 


6 weeks 


Home visit 


Remote review of Teleiiealtli parameters througliout i 


5 weeks 


8 weeks 


Discliarge liome visit 


Discliarge liome visit 
Teleliealth equipment removed 
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Table 2 Care pathways and integrated research activity 

Timeline Standard COPD service Telehealth-supported COPD service 

Intervention 1 day - First home visit after hospital Home visit (Trial info pack) Home visit (Trial info pack) 

discharge (baseline time 0) 

3 days Home visit: 

Consent 

Baseline SGRQ (paper) 
Baseline EQ-5D (paper) 
GP record diaries (paper) 
Letter to inform GP of trial 



5 days 


Home visit 


Telehealth equipment installed 
Baseline EQ-5D (device) 
GP visit data (device) 


2 weeks 


Home visit 


Remote review of Telehealth parameters 
throughout 8-week intervention 


5 weeks 


5-week EQ-5D (paper) 


5-week EQ-5D (on device) 


6 weeks 


Home visit 





8 weeks Discharge home visit: Discharge home visit: 



8-week SGRQ (paper) 8-week SGRQ (paper) 

Given materials for 6-month SGRQ Given materials for 6-month SGRQ 

Given monthly EQ-5D and GP visit diaries Given monthly EQ-5D and GP visit diaries 

Telehealth equipment removed 
Measurement of outcomes (postal return): 
Completion of 6-month SGRQ 
Completion of monthly EQ-5D 
Completion of monthly GP record visit diaries 
EQ-5D, EuroQol 5 Dimensions; GP, General practitioner; SGRQ, St. George's Respiratory Questionnaire. 



Home visit: 
Consent 

Baseline SGRQ (paper) 
Letter to inform GP of trial 



Follow-up 8 months (6 months after discharged Measurement of outcomes (postal return): 

from service) _ , . , , 

Completion of 6-month SGRQ 

Completion of monthly EQ-5D 

Completion of monthly GP record visit diaries 



subsequent 6-month follow-up period after being dis- 
charged from the service (total trial time frame of 
8 months). It was not possible to blind the involved parties 
due to the complex nature of the intervention and the 
study design. 

The COPD discharge team were responsible for col- 
lecting completed self-report data during the 8-week 
intervention. Although it was recognised that this ap- 
proach would need to be carefully monitored, clini- 
cians were involved in data collection for a number of 
reasons: 

• The nursing team were concerned that patients may 
become stressed if receiving multiple visitors (e.g., 
COPD nurse plus research nurse) whilst recovering 
from severe illness; 

• The commissioning team wished to generate robust 
evidence but had limited funds with which to do 
this; 

• One of the remits of the funding body 
(Collaboration for Leadership in Applied Health and 
Research Care for South Yorkshire) was to involve 



multiple partners in research, including frontline 
clinical staff, in order to bridge the gap between 
research and implementation in healthcare [19]. 

The clinical team underwent Good Clinical Practice 
(GCP) training and were carefully instructed in the re- 
quirements of the protocol. 

Inclusion/exclusion criteria 

The recruitment inclusion/exclusion criteria were as 
follows: 

Inclusion criteria 

• Between one and three previous admissions 
(including the current admission) in the previous 
12 months from the date of discharge with COPD 
as the primary or secondary documented reason for 
hospitalisation; 

• Referred to the COPD discharge service; 

• Willing to consider using Telehealth as part of the 
discharge plan; 
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• Able to communicate in English and read English (a 
requirement of the technology); 

• Have a telephone landline in the home (a 
requirement of the technology). 

Exclusion criteria 

• Cognitive impairment to the extent that it impedes 
ability to participate; 

• Other significant impairment(s) which restrict ability 
to participate; 

• Existence of co-morbidities which require ongoing 
intervention from other community services; 

• More than three hospital admissions for COPD 
within the prior 12 months; 

• General practitioner (GP) identifies that person is 
unsuitable to participate (e.g., due to a mental health 
condition which could affect outcome 
measurements). 

Data collection 

Table 2 provides more detail on trial pathways and data 
collection time points. The main planned sources of data 
collection were: 

1. Researcher-collected data: demographic information 
(age and sex) were collected by the research team 
from routine referral records. Extracts from the 
Secondary Uses Service (SUS (http://www.hscic.gov. 
uk/sus)) database, which provides data on hospital 
readmissions, were provided by the (blinded) PCT 
statistical team to the research team for each 
participant for the 8 months that they participated 
in the trial. 

2. Clinician-collected data: the St. George's Respiratory 
Questionnaire (SGRQ), a validated self-measure of 
respiratory disease-related quality of life [20], was 
completed by the participant at baseline and 8 weeks, 
and was to be overseen and collected by the visiting 
clinician. SGRQ was also completed at 8 months 

(6 months post-intervention) and returned to the 
research team via post. This was supplemented by 
members of the clinical team calling participants to 
remind them to complete and send their 8-month 
SGRQs; 

3. Device-collected data: a self-report patient-completed 
diary to record GP visits was recorded on paper by 
participants receiving the standard service and was 
configured to be completed on the device by those 
receiving Telehealth, during the 8-week intervention. 
During the 6-month follow-up, participants in both 
trial arms were asked to complete monthly paper 
diaries recording their GP visits. The EuroQol 5 
Dimensions questionnaire (EQ-5D-3 L), a widely used 



measure of health outcomes standardised in a wide 
variety of conditions [21], was embedded into the GP 
diaries for those allocated to the Standard service 
and into the device for participants with the 
Telehealth-supported service. 

Outcome measures 

The two co-primary outcome measures were: 

1. The proportion of participants re-admitted to 
hospital with COPD during the 8-week intervention 
and 6-month follow-up, determined using SUS data 
on hospital readmissions; and 

2. Change in self-reported health-related quality of life 
at baseline, 8 weeks, and 6-month follow-up through 
application of the SGRQ [20]. 

The secondary outcome measures were: 

1. The proportion of patients requiring unscheduled 
healthcare support for the 8 week intervention 
period and 6-month follow-up, determined through 
analysis of SUS data and patient-completed diaries 
of GP visits; and 

2. Cost-effectiveness through quality adjusted life years 
(QALYs) estimated from analysis of EQ-5D-3 L [21] 
data, GP visit data, SUS data, and the COPD 
discharge team's patient contact records. 

Analysis - primary outcome measures 

Analysis was conducted on an intention-to-treat basis. 
For the primary outcome measures, descriptive statistics 
are presented. No inferential statistical analyses were 
performed as the main objective of the study was to as- 
sess the trial methodology. 

Analysis - cost-effectiveness and cost utility 

Data on hospital admissions and unscheduled healthcare 
support, EQ-5D scores, and COPD team patient contact 
records were to be accessed and analysed to provide an 
indication of the cost-effectiveness of each care pathway. 
A cost-utility analysis was carried out based on the esti- 
mated costs of Telehealth equipment, installation and 
de-installation of units, telemonitoring, and clinician 
costs. The cost of a day of COPD-related hospital admis- 
sion and other secondary care services, including acci- 
dent and emergency (A&E) visits, were extracted from 
the National Reference Costs Publication [22]. 

Ethics 

All necessary NHS ethical and governance approvals 
were obtained from the South Yorkshire Research Ethics 
Committee (reference 10/H130/48) and from the rele- 
vant PCT. 
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Results 

Throughout the trial, recruitment rate and quality of 
data collection were significantly impacted by an un- 
anticipated reduction in clinical staff capacity. These fac- 
tors precluded progression to a full RCT in this setting. 
The findings in relation to the three objectives of the 
pilot RCT are outiined below. 

Objective one: trial methodology 
Recruitment 

The CONSORT [15] flow chart for the pilot trial is shown 
in Figure 1. Recruitment and intervention delivery took 
place from November 2010 to December 2011, with 
follow-up for 6-month post-intervention data capture 
continuing to June 2012. Although the recruitment target 
of n = 60 was achieved, this was attributed to the pilot 
RCT time frame being extended from 3 to 14 months due 
to slow recruitment to the study. During the trial time 
frame, 450 patients were referred to the COPD discharge 



' \ 

Enrollment 

■ 



service, of which 270 (60%) met the inclusion criteria. Of 
these, 132 could not be considered for inclusion in the 
study due to reasons other than declining participation, as 
summarised in Table 3. Some of the most common rea- 
sons were lack of clinician resources due to staff attrition 
that resulted in a long waiting list for admission to the ser- 
vice (thereby breaching the care and research protocol), 
immediate hospital readmission, and patient holidays/holi- 
day periods. Of the remaining patients, 75 dechned to par- 
ticipate and 63 (14%) agreed to enter the study and were 
randomised. Trial acceptance rate was 45.7% (out of 138 
patients who were eligible and were not excluded for other 
reasons); 31 participants were randomised to the standard 
service and 32 participants were randomised to receive 
the Telehealth-supported service. The clinicians reported 
that refusal to participate was most often due to the per- 
son feeling too unwell. 

Of the 63 randomised participants, data for 10 (15.9%) 
were excluded from analysis as data completion was 



Assessed for eligbility (n=450) (100%) 



Excluded (n=387) (66%) 

• Not meeting Inclusion criteria (n=180) (40%) 

• Declined to participate (n=75) (16 7%) 

• Other reasons (n=132) (29.3%) 



Randomized (n=63) (14%) 



Allocation 



/AJIocatedto Control (n=31) (6 9%) 

♦ Received allocated intervention (n=31) (5 9%) 



Allocated to Tetehealth (n=32) (7.1 %) 

♦ Received albcated intervention (n=32) (7.1%) 



Fol low-Up 



Lost to follow-up (missing data) (n=6) (1 3%) 
Discontinued Intervention (rH3) (0%) 



Lost to follow-up (missing data) (n=4) (0.9%) 
Discontinued Inlewentlon (n^) (1 1%) 



Analysis 



Analysed (n=25) (5 6%) 

• Excluded from postSwk nterfention analysis 

(deceased) (n=1)(0.2%) 



Analysed (n=23) (5 1%) 

• Excluded from post 8wk Intervention analysis 

(deceased) (n=1) (0.2%) 



Figure 1 CONSORT flow chart. 
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Table 3 Reasons for exclusion from trial 



Reasons (n = 1 32) Number 

(% of eligible patients) 

Backlog on telephone referral waiting list 59 (25.5%) 

Not seen within adequate trial time frame 15 (5.5%) 

Readmitted to hospital straightaway 11 (4.1%) 

Discharged over Christmas holiday period 10 (3.7%) 

Disruptions to care pathway schedule 6 (2.2%) 

Unable to contact 5 (1 .9%) 

Patient does not believe they have COPD 4 (1 .5%) 

Care home resident 3(1.1%) 

Unknown clinical reason 3(1.1%) 

Going on holiday 2 

Discharged elsewhere 2 

Leaving area 1 

Offered trial previously 1 



inadequate. A further 5 participants randomised to the 
Telehealth-supported service were lost to follow-up 
(Table 4). In three instances, this was due to unexpected 
problems with technology connectivity despite the exist- 
ence of a protocol to screen out such problems. One 
participant in each group died prior to completing the 
6-month follow-up. 

Protocol adherence and success of data collection strategy 

Table 5 summarises the completeness of data collection 
at each time point and provides an indication of extent 
of adherence to the research protocol. A total of 83.3% 
of participants completed the intervention within the 8- 
week service delivery time frame. 

After 6 months of data collection an audit was con- 
ducted to determine adherence to data collection proce- 
dures. Significant gaps were identified which resulted in 
the initiation of additional procedures. The most signifi- 
cant change was the introduction of a research nurse in 
month 10 to take consent and collect trial-related data, 
thereby relieving the clinical team of this responsibility. 
Protocol adherence during the 8-week intervention in- 
creased to 100% after the research nurse was involved. 
As demonstrated in Table 5, there were many instances 
of missing or invalid SGRQ data, with 56.6%, 54.2%, and 
28.3% valid completion rates for each respective time 
point (when excluding drop-outs and missing consent). 

Table 4 Reasons for discontinuing Telehealth intervention 



Reason Number 

No landline 2 (0.4%) 

Unable to install 1 (0.2%) 

Refused unit at installation stage 1 (0.2%) 

Found unit difficult to use 1 {02%) 



Only 43.8% of participants had a valid SGRQ score for 
both baseline and 8-week time points, and this figure re- 
duced to just 8.7% for all three time points. The SGRQ 
is designed to be completed by the participant overseen by 
a researcher or healthcare professional. Although this was 
agreed in the protocol, clinicians tended to leave the SGRQ 
with participants to complete in their own time. Clinician 
feedback during the pilot was that they did not feel com- 
fortable overseeing SGRQ completion as they were wary of 
biasing participants' responses when asked for advice. 

The completion rate for the standard service 8-week 
self-report diaries (which included the EQ-5D) was simi- 
larly challenged. The standard service group EQ-5D 
completion rate at baseline was 72.0% with a 5 week 
completion rate of 44.0%, which did not allow compari- 
son with EQ-5D data from the Telehealth-supported 
group (embedded within Doc@Home). The self-report 
diaries for the standard service group were completed in 
44.0% of cases. The postal return of monthly diaries in 
the 6-month follow-up (which were given to both 
groups) did not yield meaningful data (12.5% returned). 

Objective two: healthcare usage and sample size calculation 
Baseline characteristics 

Table 6 shows age and sex distribution for the 53 consented 
participants. The Telehealth-supported group contained a 
greater proportion of males and had a slightly higher mean 
age. Lack of comparability between the two trial arms was 
a likely consequence of the small sample size. 

Healthcare service usage data 

Data on the frequency and length (bed days) of hospital 
admissions, frequency of A&E visits (which did not lead 
to hospital admission), and frequency and type of 
community healthcare service contacts (other than the 
COPD discharge service) were extracted for all partici- 
pants (who completed the 8 week intervention) for the 
duration of the intervention and 6-month follow-up. Re- 
sults summarised in Table 7 show that participants re- 
ceiving the standard service had a lower readmission 
rate, fewer hospital admissions, and fewer inpatient bed 
days than those receiving the Telehealth-supported ser- 
vice. Frequency of community healthcare service contact 
was similar between the two groups. It was not possible 
to infer the number of GP visits from the available data. 

Sample size calculation 

It was not possible to use the pilot RCT data to generate 
a sample size calculation due to incomplete data collec- 
tion. Therefore, the calculation was conducted using es- 
timates of readmission rates, clinically meaningful effect 
sizes based on the literature, and clinical expertise. A 10 
to 20% relative reduction in hospital readmission rate 
was deemed to be clinically meaningful. Table 8 shows 
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Table 5 Data completion for research outcomes 





Standard 


Telehealth 


Total 


Total randomised 


31 (100%) 


32 (100%) 


63 (100%) 


Consent 








Valid consent 


25 (80.6%) 


28 (87.5%) 


53 (841%) 


Missing consent 


6 (1 9.4%) 


4 (12.5%) 


1 0 (1 5.9%) 


Dropped out 








Prior to 8-weel< completion 


0 (0.0%) 


5 (15.6%) 


5 (7.9%) 


Prior to 8-montli completion 


1 (3.2%) 


1 (3.1%) 


2 (3.2%) 


Intervention length 








8 weeks 


20 (64.5%) 


20 (62.5%) 


40 (63.5%) 


Less than 8 weeks 


0 (0.0%) 


0 (0.0%) 


0 (0.0%) 


More than 8 weeks 


5 (16.1%) 


3 (9.4%) 


8 (1 2.7%) 


Baseline SGRQ 








Valid 


16 (51.6%) 


14 (43.8%)* 


30 (47.6%) 


Invalid** 


5 (16.1%) 


3 (9.4%) 


8 (1 2.7%) 


Missing 


4 (1 3.0%) 


7 (21.9%) 


11 (17.5%) 


8-week SGRQ 








Valid 


11 (35.5%) 


1 5 (46.9%) 


26 (41 .3%) 


Invalid 


2 (6.5%) 


1 (3.1%) 


3 (4.8%) 


Missing 


1 2 (38.7%) 


7 (21.9%) 


1 9 (30.2%) 


6-month SGRQ 








Valid 


8 (25.8%) 


5 (15.6%) 


1 3 (20.6%) 


nvalid 


1 (3.2%) 


1 (3.1%) 


2 (3.2%) 


Missing 


5 (16.1%) 


5 (15.6%) 


1 0 (1 5.9%) 


Had not reached time point by Jan 12 


1 0 (32.3%) 


11 (34.4%) 


21 (33.3%) 


Participants with valid baseline & 8-week SGRQ data 


9 (29.0%) 


1 2 (37.5%) 


21 (33.3%) 


Participants with valid baseline, 8-week & 6-month SGRQ data 


2 (6.5%) 


2 (6.3%) 


4 (6.3%) 



*One participant who dropped out completed the baseline SGRQ; **Not enough completed questions to generate a valid score. 
SGRQ, St. George's Respiratory Questionnaire. 



the sample sizes required to detect reductions in this 
range, given 90% power to detect significant differences at 
a P value of 0.05. If we take an intermediate value, with a 
15% relative reduction in hospital admissions from 34% to 
29%, 1,517 patients per arm (n = 3,034 total) would be re- 
quired for a full RCT. 



Objective tliree: quality of life and preliminary evaluation 
of cost-effectiveness 

SGRQ data analysis shows that both groups reported an in- 
crease in disease-related quality of life (decrease in SGRQ 
score) between baseline and 8 weeks (Table 9). However, 
this increase was larger in the standard service group. 



Table 6 Baseline demographic information 



Demographic 



Standard 



Telehealth 



Total 



Sex 












Male, n (%) 


7 (28.0%) 


1 2 (42.9%) 


1 9 (35.8%) 




Female, n (%) 


18 (72.0%) 


16 (57.1%) 


34 (64.2%) 


Age (years) 












Mean (SD) 


65.88 (9.39) 


67.22 (1 1 .60) 


66.59 (10.54) 




Median (IQR) 


68.00 (58.85-72.85) 


70.80 (60.93-73.73) 


69.60 (59.80-73.40) 




Minimum 


41.2 


44.2 


41.2 




Maximum 


78.4 


90.9 


90.9 
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Table 7 Healthcare service use by participants 

Standard Telehealth Total 

48 

12 

25.0% 

23 
0.48 
5 

150 
3.13 
43 

5 

10.4% 
6 

0.13 
2 




21 

0.44 
4 

2 

4.2% 

Due to gaps in data collection the analysis method was 
adjusted so that EQ-5D scores were calculated from par- 
ticipants' SGRQ scores using a mapping formula pro- 
duced in a previous study in patients with COPD [23]. 
This estimates EQ-5D score as a function of SGRQ total 
score and sex. Quality adjusted life years (QALYs) were 
calculated from the EQ-5D scores using the trapezium 
rule. Missing data were imputed using the last 

Table 8 Required per arm sample size at P = 0.05 and 0.9 
power 



Relative 


Absolute reduction 


Sample size 


reduction 


in number of admissions 


per arm 




per 100 discharges 




10% 


3 


4,240 


15% 


5 


1,517 


20% 


7 


768 



observation carried forward method. Costs and QALYs 
were calculated for each group, and then used to plot 
data on the cost-effectiveness plane and to produce asso- 
ciated cost-effectiveness acceptability curves. A value of 
£20,000 per QALY was used to determine the probability 
that the intervention is cost-effective under current 
funding conditions. 

The primary analysis was based on all NHS costs and 
was performed using estimates of unit costs and esti- 
mates of resource use. There was a higher mean total 
cost in the Telehealth-supported group (£1,750 vs. £580 
for the standard service). Comparison with the mean 
cost difference showed an incremental cost per QALY 
gained of £68,811 (Table 10). 

A secondary sensitivity analysis was carried out based 
on just the costs relating to community care, i.e.. Tele- 
health and nursing contacts. The rationale for this was 
that hospitalisations had a disproportionate effect on the 
results, and being so rare in a pilot study, their mean ef- 
fect was possibly largely due to chance. 

The results of the secondary analysis, using only the 
costs of community care, showed that there is a 71.4% 
chance that the Telehealth-supported services are cost- 
effective given the willingness to pay of £2,041 per 
QALY gained (Table 11). 

To summarise, when considering community care 
costs only, and estimated equipment costs of £455 over 
5 years with a fairly low utilisation rate (three users per 
year), the Telehealth intervention is perceived to be 
cost-effective. However, when hospital admissions data 
are included in the analysis then the Telehealth interven- 
tion is not deemed to be cost-effective. 

Discussion 

In this paper we describe a pilot randomised controlled 
trial of a Telehealth intervention for COPD. The results 
showed that it would not be feasible to continue the 
pilot trial to a full RCT. In conducting this research we 
have identified issues of critical importance for any sub- 
sequent study of this complex intervention, including 
the involvement of clinicians in recruitment and in the 
research process. The service commissioners' expect- 
ation was that Telehealth-supported services might be 
clinically and cost-effective for people receiving time- 
limited support following hospital discharge. Although 
the pilot trial has not been able to give a robust indica- 
tion of clinical or cost benefit due to its small sample 
size and incomplete data collection, it has achieved its 
objective of demonstrating the practicalities of answering 
these questions with a full scale RCT in this particular 
setting. 

Compared to studies recruiting participants at mul- 
tiple sites and working with several clinical teams, it 
could be presumed that working with one clinical team 



Sample size 25 23 
Proportion with hospital admissions 

Number 4 8 

Percentage 1 6.0% 34.8% 
Number of hospital admissions 

Total 7 1 6 

Mean 0.28 0.70 

Maximum 3 5 
Total number of hospital days 

Total 21 129 

Mean 0.84 5.61 

Maximum 14 43 
Proportion with A&E visits 

Number 4 1 

Percentage 1 6.0% 4.3% 
Number of A&E visits 

Total 5 1 

Mean 0.20 0.04 

Maximum 2 1 
Proportion with community nurse contacts 

Number 4 5 

Percentage 16.0% 21.7%) 
Number of community nurse contacts 

Total 8 1 3 

Mean 0.32 0.57 

Maximum 3 4 
Mortality rate 

Number 1 1 

Percentage 4.0% 43%> 
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Table 9 St George's Respiratory Questionnaire (SGRQ) analysis for participants with valid baseline and 8-week SGRQ 
scores 



Time point 


IVletric 


Standard (n = 9) 


Telehealth (n = 12) 




Mean (SD) 


65.34 (13.68)* 


69.63 (16.58) 


Baseline (n = 21) 


Median (IQR) 


67.24 (52.97-77.73) 


73.32 (55.16-83.85) 




Min - Max (Range) 


44.77-82.25 (37.48) 


35.88-88.09 (52.21) 




Mean (SD) 


59.18 (13.21) 


67.47 (15.70) 


8-week (n = 21) 


Median (IQR) 


62.14 (44.52-67.98) 


69.99 (55.82-76.83) 




Min - Max (Range) 


42.71-81.66 (38.95) 


37.18-87.88 (50.70) 



*The SGRQ is scored from 0 (best possible hiealth status) to 100 (worst possible health status). 



with high volume referral rates in one community would 
simplify the research process. The study was conducted 
within an existing clinical service, thus bringing in- 
creased external validity through its pragmatic design 
[24] and the ability to directly inform one region's 
commissioning choices with regard to Telehealth. How- 
ever, the process of undertaking this study reinforces 
findings from larger-scale studies (e.g., Hendy et al. [25]) 
regarding the difficulty of evaluating novel, complex in- 
terventions such as Telehealth, especially when trying to 
assess the utility of the intervention within a clinical ser- 
vice alongside undertaking rigorous research. 

The study was dependent upon one small clinical team 
working against a backdrop of NHS restructuring. Even 
though initial difficulties with obtaining staff 'buy in' 
were identified and appeared to be resolved during the 
feasibility study, incremental demands created by the re- 
search combined with the loss of a key champion for the 
trial amongst the front line staff had a deleterious effect 
over time. Further staff attrition resulted in an eventual 
total loss of 60% of staff capacity within the frontline 
clinical team. Thus, the main challenge of involving 
frontline clinicians in the research and in data collection 
was that, understandably, clinical priorities always came 
first, and when resources were stretched there was little 
room for the rigorous research processes which were re- 
quired for the pilot trial. PCT budgetary restrictions pre- 
vented return of the COPD team to its initial capacity. 
The combined effect of reduced staff capacity and some 
non-compliance with trial procedures (both staff and 
participants) resulted in incomplete data collection and 



slow recruitment to the trial. This also shows that GCP 
training of clinical staff is not enough to ensure adherence 
to trial procedures, thus research processes and data col- 
lection need to be rigorously monitored throughout the 
trial. 

Despite the successful introduction of a Research 
Nurse in month 10 to consent participants and collect 
trial data, organisational changes within the PCT would 
not have allowed continuation of the pilot trial to a full 
RCT even if the limitations of the methodology were re- 
solved. One example of an organisational change which 
impacted the project was the national reorganisation of 
the NHS, meaning that PCTs were disbanded in March 
2013 and replaced by Clinical Commissioning Groups. 

The consequences of both care pathways being rela- 
tively new within what was a recently introduced service 
cannot be underestimated. Fidelity to the pathways was 
difficult to achieve at the outset and was readily compro- 
mised. The waiting list for admission to the discharge 
service, which began to build in month 4 of the pilot 
trial due to the unanticipated 60% reduction in staff cap- 
acity, is a clear example of how the care pathways were 
changed due to service imperatives which were out of 
the control of the research team. 

There were issues regarding clinician commitment to 
Telehealth and the work-based support they needed to 
deliver this new intervention efficiently and appropri- 
ately. The importance of training and on-going support 
to deliver Telehealth is now recognised [14], but was not 
available to the clinicians who participated in this study 
with the consequence that adherence easily eroded and 



Table 10 NHS cost and QALYs over 6 months 

Item Standard Telehealth Mean 

mean (n = 25) mean (n = 28) difference 

Total costs £580 £1,749.8 £1,169.8 

Quality adjusted life 0.20 0.217 0.017 

years gained 



Table 1 1 Community care costs and QALYs over 6 months 

Item Standard Telehealth Mean 

mean mean difference 

(n = 25) (n = 28) 

Total costs £348.3 £383 £34.7 

Quality adjusted 0.20 0.217 0.017 
life years gained 



ncremental 

cost-effectiveness 

ratio 



£68,81 1 per QALY 



Incremental 

cost-effectiveness 

ratio 



£2,041 per QALY 
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confidence waned. Problems with device connectivity for 
some of the participants randomised to receive Telehealth 
despite application of the study's inclusion/exclusion cri- 
teria reinforced the lack of clinician confidence. 

Feedback from the feasibility study indicated that 
those receiving the clinician-delivered 'standard service' 
valued this service and benefitted from it, and that both 
groups expressed a preference for personalised face-to- 
face service. The question this poses is whether those re- 
ceiving Telehealth were disadvantaged in this instance 
compared with those in receipt of a specialist face-to- 
face service, which was highly tailored to patient need 
but unlikely to be sustainable in the longer term. This 
preference for face-to-face care may be more influential 
than the perceived reassurance of daily monitoring pro- 
vided through using Telehealth. Additionally, questions 
were raised by the study regarding which patients might 
gain most benefit from Telehealth. Study participants 
were recruited at the point of hospital discharge and it 
became evident that receptiveness to using Telehealth 
could be severely compromised by their illness. It is pos- 
sible that asking a patient to use a piece of technology at 
this point, even one which is simple to use, could be a 
daunting commitment to take on in addition to recover- 
ing from their exacerbation. There are unanswered ques- 
tions regarding when in the overall care pathway from 
acute to primary care can Telehealth technology be most 
effective and for how long, taking into account exacerba- 
tion severity. There are also questions around the repre- 
sentativeness of the sample, and when in the COPD 
disease pathway deployment of Telehealth would be of 
most benefit. Forty percent of patients referred to the 
COPD service failed to qualify for the trial on one or 
more of the eligibility criteria, indicating that trial partic- 
ipants may not have been fully representative of the 
COPD population as a whole and that our trial design 
may not have been as 'pragmatic' as had been intended. 
The definition of between one and three previous hos- 
pital admissions, which formed a key component of the 
eligibility criteria, was based on the local definition of early 
stage COPD. However, it could be argued that a person 
with three admissions in the previous 12 months may 
have more advanced disease compared to someone with 
one or no admissions. To summarise, commissioners and 
clinicians require greater guidance for deployment. 

By definition, complex interventions are difficult to de- 
fine, standardise, and measure, and enthusiasm to 
undertake research can underestimate this. The results 
obtained from this study question the viability of involv- 
ing front line clinical staff in data collection for robust 
research evaluation. Whilst it is true that clinicians do 
not need to be involved in data collection, the observa- 
tions of Bird et al. [26] provide further support regarding 
the importance of context and culture when conducting 



trials, yet this is not evidenced through on-going com- 
missioning of trials of complex interventions. 

Questions remain regarding how to most appropriately 
conduct local evaluations to inform commissioning deci- 
sions. Leykum et al. [27] suggest integration of participa- 
tory action research and randomised controlled trial 
methods to ensure that a complex intervention is ad- 
equately embedded within the setting. Arguably, this did 
occur within this study through the extensive involve- 
ment of commissioners and clinicians in study set up, 
design, and progress, and through regular meetings dur- 
ing the entire programme. As demonstrated by Hendy 
et al. [25], the drive to demonstrate population-based 
benefit through evaluation of complex interventions 
does not necessarily equate with the demands of imple- 
menting a complex intervention in situ. 

Conclusions 

We were able to complete an informative pilot RCT, 
despite service reconfiguration and slow recruitment 
rates. However, ultimately, these factors precluded pro- 
gression to a full RCT in this setting. On the basis of 
our experience in conducting this pilot study, we are 
able to recommend that a definitive trial should be 
multi-centre and aim to include 1,517 participants in 
each arm. Consent, randomisation, and data collection 
(in accordance with the protocol) should be supported 
by dedicated research staff rather than by clinicians. 
However, the study also raises a dUemma - there are in- 
dications that patients prefer face-to-face contact even 
when offered daily but remote interaction through Tele- 
health, yet this model of care delivery may not be sus- 
tainable in the current climate. 

Endnote 

^Early stage COPD was defined as "Between one and 
three previous admissions (including the current admis- 
sion) in the previous 12 months from the current date of 
discharge where COPD is the primary or secondary doc- 
umented reason for hospitalisation". 
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